<html>
<head><meta charset="utf-8"><title>Can I visit the AST before DefIds are generated? · t-compiler/help · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/index.html">t-compiler/help</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Can.20I.20visit.20the.20AST.20before.20DefIds.20are.20generated.3F.html">Can I visit the AST before DefIds are generated?</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="201482444"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Can%20I%20visit%20the%20AST%20before%20DefIds%20are%20generated%3F/near/201482444" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Can.20I.20visit.20the.20AST.20before.20DefIds.20are.20generated.3F.html#201482444">(Jun 20 2020 at 15:32)</a>:</h4>
<p>context: <a href="https://github.com/rust-lang/rust/pull/73532#issuecomment-646991250">https://github.com/rust-lang/rust/pull/73532#issuecomment-646991250</a></p>
<p>I'm looking around <code>librustc_expand/expand.rs</code> but it looks like everything here is an <code>AstFragment</code>, not part of the AST proper.</p>



<a name="201482467"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Can%20I%20visit%20the%20AST%20before%20DefIds%20are%20generated%3F/near/201482467" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Can.20I.20visit.20the.20AST.20before.20DefIds.20are.20generated.3F.html#201482467">(Jun 20 2020 at 15:33)</a>:</h4>
<p>Whoops, I missed <a href="https://doc.rust-lang.org/nightly/nightly-rustc/rustc_expand/expand/enum.AstFragment.html#method.mut_visit_with">https://doc.rust-lang.org/nightly/nightly-rustc/rustc_expand/expand/enum.AstFragment.html#method.mut_visit_with</a></p>



<a name="201484280"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Can%20I%20visit%20the%20AST%20before%20DefIds%20are%20generated%3F/near/201484280" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Can.20I.20visit.20the.20AST.20before.20DefIds.20are.20generated.3F.html#201484280">(Jun 20 2020 at 16:10)</a>:</h4>
<p><span class="user-mention" data-user-id="118594">@ecstatic-morse</span> so here's the idea I had:</p>
<ol>
<li>Expand macros</li>
<li><em>Before</em> generating the DefId for any of the AST, run <code>ReplaceBodyWithLoop</code></li>
<li>Generate DefIds</li>
</ol>
<p>This broke immediately because <code>PlaceHolderExpander</code> expects all ids to be <code>DUMMY_NODE_ID</code>. So I changed it to use <code>DUMMY_NODE_ID</code> and now <code>lower_node_id_generic</code> is panicking because it wants ids <em>not</em> to be dummies. Do you have any idea what's going on? I'm very confused.</p>
<div class="codehilite"><pre><span></span><code>// without dummy ids
$ rustc +stage1 -Z unpretty=everybody_loops priv-in-pub.rs
thread &#39;rustc&#39; panicked at &#39;assertion failed: `(left == right)`
  left: `NodeId(28)`,
 right: `NodeId(4294967040)`&#39;, src/librustc_expand/placeholders.rs:326:17
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
// with dummy ids
$ rustc +stage1 -Z unpretty=everybody_loops priv-in-pub.rs
thread &#39;rustc&#39; panicked at &#39;assertion failed: `(left != right)`
  left: `NodeId(4294967040)`,
 right: `NodeId(4294967040)`&#39;, src/librustc_ast_lowering/lib.rs:599:9
</code></pre></div>



<a name="201484900"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Can%20I%20visit%20the%20AST%20before%20DefIds%20are%20generated%3F/near/201484900" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dylan MacKenzie (ecstatic-morse) <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Can.20I.20visit.20the.20AST.20before.20DefIds.20are.20generated.3F.html#201484900">(Jun 20 2020 at 16:22)</a>:</h4>
<p>No. I think you have the right idea; we need to do the AST transformation for rustdoc before assigning DefIds, but I don't work on the frontend at all</p>



<a name="201485518"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Can%20I%20visit%20the%20AST%20before%20DefIds%20are%20generated%3F/near/201485518" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Can.20I.20visit.20the.20AST.20before.20DefIds.20are.20generated.3F.html#201485518">(Jun 20 2020 at 16:37)</a>:</h4>
<p>So iirc <code>lower_node_id_generic</code> is assigning new <code>HirId</code> to <code>NodeId</code> so it needs real node ids, not dummy ones</p>



<a name="201485656"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Can%20I%20visit%20the%20AST%20before%20DefIds%20are%20generated%3F/near/201485656" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Can.20I.20visit.20the.20AST.20before.20DefIds.20are.20generated.3F.html#201485656">(Jun 20 2020 at 16:40)</a>:</h4>
<p>Sure, but why does <code>PlaceHolderExpander</code> need dummies? Or I guess I'm not sure why it's being run at all on this code.</p>



<a name="201485780"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Can%20I%20visit%20the%20AST%20before%20DefIds%20are%20generated%3F/near/201485780" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Can.20I.20visit.20the.20AST.20before.20DefIds.20are.20generated.3F.html#201485780">(Jun 20 2020 at 16:42)</a>:</h4>
<p>let me push to a branch real quick</p>



<a name="201485803"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Can%20I%20visit%20the%20AST%20before%20DefIds%20are%20generated%3F/near/201485803" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Can.20I.20visit.20the.20AST.20before.20DefIds.20are.20generated.3F.html#201485803">(Jun 20 2020 at 16:43)</a>:</h4>
<p>I've never actually had to look at <code>librustc_expand</code> so I probably won't be able to give good advises</p>



<a name="201486028"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Can%20I%20visit%20the%20AST%20before%20DefIds%20are%20generated%3F/near/201486028" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Can.20I.20visit.20the.20AST.20before.20DefIds.20are.20generated.3F.html#201486028">(Jun 20 2020 at 16:47)</a>:</h4>
<p>WIP: <a href="https://github.com/jyn514/rust/tree/replace-bodies/earlier-pass">https://github.com/jyn514/rust/tree/replace-bodies/earlier-pass</a></p>



<a name="201486081"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Can%20I%20visit%20the%20AST%20before%20DefIds%20are%20generated%3F/near/201486081" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Can.20I.20visit.20the.20AST.20before.20DefIds.20are.20generated.3F.html#201486081">(Jun 20 2020 at 16:48)</a>:</h4>
<p>I know that recently <span class="user-mention" data-user-id="125294">@Aaron Hill</span> and <span class="user-mention" data-user-id="123856">@Vadim Petrochenkov</span> worked on macros and defids</p>



<a name="201486247"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Can%20I%20visit%20the%20AST%20before%20DefIds%20are%20generated%3F/near/201486247" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Can.20I.20visit.20the.20AST.20before.20DefIds.20are.20generated.3F.html#201486247">(Jun 20 2020 at 16:52)</a>:</h4>
<p>This is the relevant code, but I'm not sure what it's doing: <a href="https://github.com/jyn514/rust/blob/4933cb2090986866a041a9c11a4233ad0ec13bb3/src/librustc_expand/expand.rs#L516">https://github.com/jyn514/rust/blob/4933cb2090986866a041a9c11a4233ad0ec13bb3/src/librustc_expand/expand.rs#L516</a></p>



<a name="201486954"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Can%20I%20visit%20the%20AST%20before%20DefIds%20are%20generated%3F/near/201486954" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Can.20I.20visit.20the.20AST.20before.20DefIds.20are.20generated.3F.html#201486954">(Jun 20 2020 at 17:05)</a>:</h4>
<p>maybe I could move this into <code>build_reduced_graph</code> instead, just before it calls <code>collect_definitions</code>?</p>



<a name="201487808"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Can%20I%20visit%20the%20AST%20before%20DefIds%20are%20generated%3F/near/201487808" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Can.20I.20visit.20the.20AST.20before.20DefIds.20are.20generated.3F.html#201487808">(Jun 20 2020 at 17:21)</a>:</h4>
<p>nope, same error that it expected a dummy span</p>



<a name="201488089"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Can%20I%20visit%20the%20AST%20before%20DefIds%20are%20generated%3F/near/201488089" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Can.20I.20visit.20the.20AST.20before.20DefIds.20are.20generated.3F.html#201488089">(Jun 20 2020 at 17:29)</a>:</h4>
<p>I'm getting annoyed enough I might make a PR switching from <code>NodeId</code> + DUMMY_NODE_ID to <code>Option&lt;NodeId&gt;</code></p>



<a name="201490359"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Can%20I%20visit%20the%20AST%20before%20DefIds%20are%20generated%3F/near/201490359" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Can.20I.20visit.20the.20AST.20before.20DefIds.20are.20generated.3F.html#201490359">(Jun 20 2020 at 18:22)</a>:</h4>
<p>I don't understand how that would help</p>



<a name="201494632"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Can%20I%20visit%20the%20AST%20before%20DefIds%20are%20generated%3F/near/201494632" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Vadim Petrochenkov <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Can.20I.20visit.20the.20AST.20before.20DefIds.20are.20generated.3F.html#201494632">(Jun 20 2020 at 20:00)</a>:</h4>
<p>I don't think you need to run everybody-loop before DefIds are generated, it's run at the right time.</p>



<a name="201494640"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Can%20I%20visit%20the%20AST%20before%20DefIds%20are%20generated%3F/near/201494640" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Vadim Petrochenkov <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Can.20I.20visit.20the.20AST.20before.20DefIds.20are.20generated.3F.html#201494640">(Jun 20 2020 at 20:00)</a>:</h4>
<p>It just needs to keep nodes having DefIds.</p>



<a name="201494740"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Can%20I%20visit%20the%20AST%20before%20DefIds%20are%20generated%3F/near/201494740" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Vadim Petrochenkov <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Can.20I.20visit.20the.20AST.20before.20DefIds.20are.20generated.3F.html#201494740">(Jun 20 2020 at 20:02)</a>:</h4>
<p>(That's what I can say without digging too much into the problem, and I don't really want to dig too much into this.)</p>



<a name="201502630"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Can%20I%20visit%20the%20AST%20before%20DefIds%20are%20generated%3F/near/201502630" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Can.20I.20visit.20the.20AST.20before.20DefIds.20are.20generated.3F.html#201502630">(Jun 20 2020 at 23:03)</a>:</h4>
<p>This ended up not running <code>everybody_loops</code> at all, see <a href="https://github.com/rust-lang/rust/pull/73566">https://github.com/rust-lang/rust/pull/73566</a></p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>